Cybersecurity incident response and security operation system employing playbook generation through custom machine learning

ABSTRACT

A new cybersecurity incident is registered at a security incident response platform. At a playbook generation system, details are received of the new cybersecurity incident from the security incident response platform. At least some of the details correspond to a set of features of the new cybersecurity incident. A set or subset of nearest neighbors of the new cybersecurity incident is localized in a feature space. The nearest neighbors of the new cybersecurity incident are other cybersecurity incidents having a distance from the new cybersecurity incident within the feature space that is defined by differences in features of the nearest neighbors with respect to the set of features of the new cybersecurity incident. A custom playbook is created for responding to the new cybersecurity incident having prescriptive procedures based on occurrences of prescriptive procedures previously employed in response to the nearest neighbor cybersecurity incidents.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation Application under 35 USC § 120 ofU.S. patent application Ser. No. 16/594,538, entitled “CybersecurityIncident Response and Security Operation System Employing PlaybookGeneration Through Custom Machine Learning,” filed on Oct. 7, 2019,which is a Continuation Application under 35 USC § 120 of U.S. patentapplication Ser. No. 15/620,439, entitled “Cybersecurity IncidentResponse and Security Operation System Employing Playbook GenerationThrough Custom Machine Learning,” filed Jun. 12, 2017, which claims thebenefit of U.S. Provisional Patent No. 62/490,817, filed Apr. 27, 2017,all of which are incorporated herein by reference in their entirety.

TECHNICAL FIELD

The field of this invention generally relates to cybersecurity incidentresponse and security operation systems, and more particularly to suchsystems employing playbooks containing sets of “plays” or prescriptiveprocedures for responding to cybersecurity incidents.

BACKGROUND

Certain cybersecurity incident response and security operation systems,such as the IncMan incident response platform available from DFLabs,incorporate the use of playbooks to assist a user in responding to acybersecurity incident.

Playbooks are collections of manual and automated actions designed toresolve an incident or complete an investigation. For example, in theIncMan platform, sets of predefined playbooks can be provided based ondifferent industry standards. Each incident is categorized according toone or more type values, and these type values are used to match relatedplaybooks to an incident. Each playbook is described by some propertiesincluding type, category and actions that can be grouped insubcategories. Multiple playbooks can be linked to the same incident.Actions to be assigned to users can be defined into a playbook.Additionally, automatic actions for enrichment and containment of theincident can be assigned as part of each playbook.

In the IncMan platform, the user can select from a number ofpre-constructed playbooks and later customize the playbooks. Theplaybooks are proposed based only on the category of the incident in alog record (so no other incident fields of the log record are taken intoaccount) and any subsequent user customization (i.e., addition andremoval of actions) is not taken into account in the proposing ofpre-constructed playbooks.

Previous patent applications by the inventor of the present applicationpertaining to cybersecurity incident response systems and digitalevidence control systems include U.S. patent application Ser. No.11/784,794, filed Apr. 10, 2007, and published as U.S. PatentPublication 2008/0098219, and U.S. patent application Ser. No.14/521,328, filed Oct. 22, 2014, and published as U.S. PatentPublication 2016/0044061. The entire disclosure of both of theseprevious patent applications by the inventor of the present applicationis hereby incorporated herein by reference. It is contemplated that thesystem described herein may be used in connection with cybersecurityincident response and security operation systems employing the subjectmatter described in the above-referenced previous patent applications bythe inventor.

In certain platforms, it is known to provide playbooks that, oncecreated, can be improved in a democratic fashion by team members overthe course of time.

SUMMARY

It is an object of the invention to enhance selection of an appropriateplaybook for a new incident.

According to the apparatus and method of responding cybersecurityincidents of one aspect of the invention, a new cybersecurity incidentis registered at a security incident response platform. At a playbookgeneration system, details are received of the new cybersecurityincident from the security incident response platform. At least some ofthe details correspond to a set of features of the new cybersecurityincident. A set or subset of nearest neighbors of the new cybersecurityincident is localized in a feature space. The nearest neighbors of thenew cybersecurity incident are other cybersecurity incidents having adistance from the new cybersecurity incident within the feature spacethat is defined by differences in features of the nearest neighbors withrespect to the set of features of the new cybersecurity incident. Acustom playbook is created for responding to the new cybersecurityincident. The custom playbook has one or more prescriptive procedures,for responding to the new cybersecurity incident registered by thesecurity incident response platform, that are based on occurrences ofprescriptive procedures previously employed in response to the nearestneighbor cybersecurity incidents, and on distances of the nearestneighbors of the new cybersecurity incident in the feature space. A userof the security incident response platform is presented with the customplaybook containing the one or more prescriptive procedures forresponding to the new cybersecurity incident. The user of the securityincident response platform initiates the one or more prescriptiveprocedures contained in the custom playbook to respond to the newcybersecurity incident.

This aspect of the invention makes it possible to employ a custommachine learning system to propose a playbook tailor-made for the newincident based on all data available at any time about the incident andsimilar incidents (including all relevant fields of the new incident andall past pairings of similar incidents with playbooks), or based on atleast some of the available data. The custom machine learning system isable to learn a model based on historical data. This model can integratepast decisions (i.e., past selections of playbooks playbookcustomizations) of the users and for each new incident, it can provide anumber of playbooks based on the model.

According to the apparatus and method of another aspect of theinvention, the user of the security incident response platform respondsto the cybersecurity incident by initiating a set of one or moreprescriptive procedures that differs from the set of the one or moreprescriptive procedures contained in the playbook. At the playbookgeneration system, the new cybersecurity incident is recorded in thefeature space, and the new cybersecurity incident is automatically tiedto the one or more prescriptive procedures actually initiated by theuser in responding to the new cybersecurity incident, therebyautomatically altering subsequent recommendations of playbooks forresponding to cybersecurity incidents having features similar oridentical to the set of features of the new cybersecurity incident.

This aspect of the invention makes it possible to integrate usercustomization, as well as user removal or addition of actions afterplaybook selection, as an automatic feedback into the system such thatthe system knows which actions of the playbook were correctly proposedand which were not.

The details of various embodiments of the invention are set forth in theaccompanying drawings and the description below. Numerous other featuresand advantages of the invention will be apparent from the description,the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of the architecture of a playbook generationsystem according to the invention interfacing with a cybersecurityincident response system.

FIG. 2 is a table of fields of a log record of an incident, along withtheir associated feature types and the encoding employed for eachfeature type in order to transform the incident into a set of featuresin a feature space.

FIG. 3 is a two-dimensional representation of a set of incidents in amany-dimensional feature space.

FIG. 4 is the two-dimensional representation of FIG. 3, with concentriccircles illustrating distances within the feature space with respect toa particular new incident.

FIG. 5 is the two-dimensional representation of FIG. 4, together withrepresentations of playbooks for K nearest neighbors of the particularnew incident.

FIG. 6 is a plot of an age function used in accordance with theinvention as part of the process of determining relevance of a priorincident with respect to a new incident.

FIG. 7 is a flowchart detailing the steps corresponding to identifying Knearest neighbors and then applying the different machine learning stepsto the K nearest neighbors.

FIG. 8 is a diagram illustrating the exchange of information between auser, a cybersecurity incident response platform, a playbook generationsystem according to the invention, and a database.

FIG. 9 is a screen display through which a user can notify acybersecurity incident response system of the creation of an incident.

FIG. 10 is a screen display through which a custom playbook is presentedto a user.

FIG. 11 is a screen display through which a different custom playbook ispresented to a user.

DETAILED DESCRIPTION

The present invention provides a system designed to stand independentlyon a platform of a cybersecurity incident response system, such as, forexample; the IncMan incident response platform available from DFLabs.

The principal capabilities of the custom machine learning systemprovided by the invention can include the following:

1) Ability to provide a playbook (i.e., a set of actions) based on allrelevant fields of the new incident and based on all past pairings ofincident with playbook(s);

2) Ability to integrate user feedback, i.e., removal or addition ofactions after playbook selection;

3) Ability to respond in real time, i.e., provide playbook for a newincident almost instantly;

4) Ability to work with sparse or limited amount of data; and

5) Ability to consider Common Event Format (CEF) fields for playbookrecommendation.

The machine learning concept is based on the principle of looking ofpast incidents that are similar to a new incident. Once such similarincidents are found, their playbooks are collected as relevant ones andreasoned over. The playbooks can be also represented as sets of actions.Then the frequency with which specific actions are included in allplaybooks is considered. Based on the frequency of action presence anddistance of relevant playbooks, a weight between 0 and 1 is assigned toall actions present in the platform of a cybersecurity incident responsesystem.

Subsequently, user feedback is integrated: actions that were previouslyrejected by the user are less likely to be proposed and actions thatwere previously manually added (so-called custom actions) are consideredfor addition.

Finally, given a threshold (i.e., a number between 0 and 1), a playbookis proposed with all actions with weight above the threshold. By giving(or predefining) multiple thresholds, the system can propose multipleplaybooks simultaneously.

With reference to FIG. 1, one embodiment of a playbook generation system10 according to the invention is accessed by a cybersecurity incidentresponse system 12 through a predefined application program interface(API) under the control of API controller 14, and responds to a numberof requests generated by cybersecurity incident response system 12. Forexample, cybersecurity incident response system 12 makes REST calls toAPI controller 14 to notify playbook generation system 10 of incidentscreated, updated, or closed, customizations of source incidents, use ofgenerated playbooks, and additions or removals of actions fromplaybooks; and, API controller 14 makes REST calls to cybersecurityincident response system 12 to notify it of proposed playbook actions.The playbook generation system 10 is able to autonomously construct andupdate the model independently of the platform of cybersecurity incidentresponse system 10.

The playbook generation system 10 accesses the data in databases 16 and18 through its connectors 20 and 22 and gets all relevant data forbuilding the model from the databases (both MongoDB database 16 andPostgreSQL database 18—the two databases utilized by the cybersecurityincident response system 12). Mongo connector 20 defines how data iseither read from or stored to MongoDB database 16. Incident connector 22defines how data is either read from or stored to PostgreSQL database18. An incident is represented as a row in an incidents table in thePostgresSQL database. To be able to convert the columns of the table tofeatures in feature transformer 24 we first need to know the types ofthe columns. These types are fetched from an ims_fields table. Also,some of the columns of the table are encrypted. This information isobtained from an information schema table. Information from all thesethree tables combined allows us to load the stored data to playbookgeneration system 10 and further parse the data.

Feature transformer 24 encapsulates all of a set of feature generators,details of which are described below. Feature transformer 24 provides anapplication program interface that, given the parsed data from theincidents table described above, produces a Boolean matrix of extractedfeatures, Each feature generator is a transformation that for a givenvalue from an incident and its type produces one or more features. Thefollowing feature generators are implemented: Boolean feature generator,present feature generator, enum feature generator, and enum set featuregenerator. The operation of these feature generators is described belowin connection with FIG. 2.

The playbook generation system 10 saves its model and performancemeasurements into several MongoDB Collections. The playbook generationsystem itself is exposed through a web server providing RESTimplementation of the API.

The playbook generation system follows the standard machine learningapproach: model construction, model query and model update. In the firststage a model is constructed based on the historical data. In a querystage the model is queried for an output (i.e., a recommended playbook).In the update stage, the model is updated with new information.

The machine learning technique is based on the concept of fuzzy nearestneighbor search and subsequent data fusion. An example of fuzzy nearestneighbor search is described in Keller, James M., Michael R. Gray, andJames A. Givens, “A fuzzy k-nearest neighbor algorithm,” IEEEtransactions on systems, man, and cybernetics 4 (1985): 580-585(ieeexplore.ieee.org/document/6313426), which is hereby incorporatedherein by reference.

Model Construction

The main steps of the model construction stage are as follows:

1) Extraction of relevant attributes and translation into features; and

2) Transformation of historical (closed) incidents into the featurespace; and

3) Storage of the populated feature space; and

4) Storage of custom actions played for each incident.

The transformation of features of an incident into a feature space (byfeature transformer 24 in FIG. 1) involves creation of a feature vector,which is an n-dimensional vector of numerical features that representthe incident. The feature space is the vector space associated with thefeature vectors.

Model Query

The model is queried by requesting playbook for a new incident. The mainsteps are the following:

1) Transformation of the new incident into the features;

2) Localization of K nearest neighbors (special cases for highlyirregular feature spaces are described below);

3) Fusion of: frequencies of action played, distance of neighbors fromthe incident, age of closed incidents, and user feedback: and

4) Thresholding of the action set to get the playbook.

Model Update

The model is updated once a particular new incident is closed. Thismeans that the following steps (similar to the model construction stage)are performed:

1) Transformation of the closed incident into the feature space;

2) Update of the stored feature space; and

3) Update of the custom action storage.

Feature extraction and feature space representation

With reference to FIG. 2, the features for an incident are generatedfrom fields, each of which has a certain type. Depending on the type, aspecific transformation is called.

The schema of the incidents table allows values in the columns to benull. Feature “present” is true if the value is not null, which isparticularly useful for rich fields; such as texts or URLs. It would notbe possible to create a feature for all possible URLs, because there aretoo many of them. But it might be interesting to know whether aninvestigator had to deal with any URL, because consequently theinvestigator might take some action based on this fact. On the otherhand, “checkbox” is a very simple type, which is either checked orunchecked, A checkbox cannot have a missing value, Thus the “Boolean”feature directly reflects the Boolean state of the checkbox.

The table of FIG. 2 illustrates the transformation of different types offields into features:

Basically, all different types are separated into four groups:

1) Untranslatable into the feature space (Feature=None). For example,the “text” type is too rich to translate, and it is difficult to extractmeaningful representative features out of it. Other features might justnot be robust, as, for example, date and time. Since it is not possibleto know the context, it is not possible to guess that a particular hourof a day of an incident is going to be representative when comparingincidents.

2) Boolean encoding—i.e., types that are either Boolean by nature orhave information that is tested for presence—i.e., present or missing.The test on presence is reduced to Boolean representation as well. Forexample, the types IP, HASH, EMAIL, URL, and FILENAME are too rich forone-hot or many-hot encoding, and yet it might be useful to know that aninvestigator had to deal with fields of these particular types.

3) One-hot encoding—a special type of encoding for types that contain afinite set of possible values. For example, for type “timezone,” itmight be interesting to know that the problem being solved occurred in aspecific part of the world. Given a field that can have only one ofthree values: {A, B, C}, we can encode the values into binary vectors001, 010, and 100 (i.e., only one bit from the binary encoding is 1,thus one-hot encoding). These binary vectors are then added into thefeature space. Thus, for types with, e.g., three values, the featurespace will be extended by 3 dimensions.

4) Many-hot encoding—a generalization of one-hot encoding is many-hotencoding, which allows multiple positions of a vector to be true.Many-hot encoding is a generalization of a one-hot encoding to representsubsets of elements.

For Boolean encoding, a Boolean feature generator of feature transformer24 of FIG. 1 extracts a Boolean value from a given input as implementedby the python's bool._call_function. For example:

-   -   “True”->True    -   “False”->False    -   “ ”->False    -   0->False    -   1->True

A present feature generator of the feature transformer produces feature“True” if the underlying value is present (is not null or is not anempty string, and produces feature “False” otherwise.

An enum feature generator of the feature transformer encapsulatesone-hot encoding. To encode a variable having one of N possible values,a Boolean vector of length N is produced. Each position of the vectorrepresents exactly one value from the original set of N values. Only onevalue of the vector is allowed to be “True.” Also, the value being“True” signifies that the element represented by the particular positionof the vector is the one assigned to the variable. A generalization ofone-hot encoding is many-hot encoding, which allows multiple positionsof a vector to be true. Thus, the vector can be represented as a subsetof the original N elements. An enum set feature generator of the featuretransformer encapsulates many-hot encoding.

With reference to FIG. 3, there is shown a representation of incidents300 in a two-dimensional representation of what is actually amany-dimensioned feature space 302. A two-dimensional projection of amulti-dimensional space is chosen only for ease of illustration. Inparticular, a number of historical incidents 300 are defined by theirattributes. These incidents are represented as samples in a so-calledfeature space 302. The features are constructed from the attributes ofthe incident. The representation of FIG. 3 will evolve in the nextsections to explain steps of the nearest neighbor search.

Nearest Neighbor Search

With reference to FIG. 4, the feature space 302 is a space over which adistance metric can be defined. With the metric, one can measure thedistance between any two points on the feature space. Given twoincidents with different attributes, they can be transformed into thefeature space (through the encoding explained above) and subsequently, ametric can be called. This metric measures the distance in the featurespace, and thus we can compare distance between any two incidents. Usingthis approach, we can find K nearest neighbors (with asymptoticallylinear complexity). In FIG. 4, the nearest neighbors of incident 304 arethose other incidents whose distance in the feature space as measured bythe metric is shortest with respect to incident 304. The concentriccircles in FIG. 4 represent different distances from incident 304 asmeasured by the metric.

Data Fusion

With reference to FIG. 5, given K nearest neighbors of incident 304 (theneighbors within the outermost circle in FIG. 5), it is possible can getall playbooks 306 played for each incident. The existing incidents inthe feature space 302 have been resolved previously and thus have one ormore playbooks 306 with actions assigned. The rectangles of playbooks306 correspond to actions. In reality, the set of rectangles correspondto all actions every played. Those actions that were committed arecolored dark. In other words, for each incident, the committed actionsfrom the complete action set are listed. The playbooks in general differin some actions, and some actions are identical for various playbooks.In FIG. 5, playbooks are represented by rectangles correspond to Booleanrepresentations of eight different actions that may or may not have beencalled for in different playbooks. As the incidents are closed, weexpect that the committed and completed actions are those that should berecommended. For each incident, we collect all committed actions. Thesecommitted actions are considered to be ideal playbooks for eachincident.

The formulas set forth below, which culminate in a score calculation fora particular action, are iterated over all actions in the action set,and for each action, there is a Boolean value of 0 or 1 depending onwhether the action was committed are not in connection with a particularincident. The actions themselves are not Boolean, however, because theyare prescriptions for the operator of what to do.

Having the set of actions to be recommended for each past incident, weconsider each action separately. For each action, we compute a scorefrom the interval [0, 1] that translates into the certainty ofrecommending this action. The following partial computations are usedfor the computation of the score:

relevance_(α)(i _(new) ,i _(ref))=1/(α·dist(i _(new) ,i_(ref))+(1−α)·age(i _(ref))+1)

I.e., first, the relevance of an incident with respect to another one isdefined by the equations above. The relevance is given by the distancebetween the incidents in the feature space and by the age of thereference incident (given that the new incident has age 0). To find Kneighbors, the relevance of each incident is measured as defined by theabove equation. Using the relevance equation, it is possible to find Knearest incidents according to a weighted metric, which is influenced bythe user's setting of the parameter α, discussed below.

The distance between two incidents is computed as the average ofweighted feature values:

${{dist}_{w}\left( {i_{1},i_{2}} \right)} = {{\frac{1}{{features}}{\sum\limits_{f \in {features}}^{\;}{{w_{f} \cdot {}}{i_{1}\lbrack f\rbrack}}}} - {{i_{2}\lbrack f\rbrack}{}_{1}}}$

The weights are set by the users in the user interface and are set to“ignore,” “low,” “medium,” and “high” corresponding to values 0, 0.25,0.5, 1 respectively. i.e., ignored fields are not considered, low weightfields are half as important as the default medium fields, and highweight fields are twice as important as default medium fields and fourtimes as important as low weight fields.

The age(i_(ref)) function is defined as:

age(i _(ref))=min(1;days(i _(ref))/τ)

I.e., it is a normalized thresholded age, where days(i_(ref)) is theincident age in days and τ is a threshold with a default value set to365. The plot of FIG. 6 shows the behavior of the age(i_(ref)) functionfor the default values.

The α parameter is from the range [0,1] and it specifies the weight thatis given to similarity and weight that is given to age (either thesimilarity of incidents is more important and the age is less importantor vice versa). By default, the recommended value is 0.5.

Finally, the score of an action is computed as an average relevanceweighted by the commitment frequency of the action:

${{score}\left( {a_{j},i_{new}} \right)} = {{\frac{1}{|n|}{\sum\limits_{i_{ref} \in {relevant}}\mspace{14mu}{{committed}\;{\left( {i_{ref},a_{j}} \right) \cdot \text{∀}}a_{j}}}} \in {A\mspace{14mu}{{relevance}_{\alpha}\left( {i_{new},i_{ref}} \right)}}}$

The score is computed over all relevant incidents (denoted as “relevant”in the sum subscript) by the multiplication of action presence (denotedby function “committed(i_(ref), a)” which returns 1 if the action a_(j)was committed in the incident i_(ref), 0 otherwise) with incidentrelevance. There are K nearest incidents (determined according to therelevance distance discussed above) that are members of the “relevant”set. The action set “A” is the complete set of actions in the securityincident response platform. The set “A” can be optimized (its sizeminimized) by defining A as a union of committed actions over allrelevant incidents. Computationally, however, the mathematicaloperations over the action set are very fast, and the asymptoticalcomplexity is linear, and so the size of the action set should notmatter.

The fuzziness of the approach described above lies in the computation ofan action score; i.e., the relevance is used to weigh the score of theaction. In other words, the fuzzy set membership is defined by therelevance and it is used to “merge” information from the K nearestneighbors together. In contrast, a classical non-fuzzy K nearestneighbor technique might omit the relevance in a score computation andthe score would be given only as the number of times the action wascommitted divided by K.

Playbook Generation Process

Once all actions have a score assigned, we can choose a threshold. Allactions with a score above the threshold will be recommended to beplayed. For multiple thresholds, multiple playbooks can be generatedautomatically as well.

Handling Special Cases

The spatial distribution of incidents within the feature space can varywidely from client to client. The following sections describe specialcases and describes, how the system handles these cases.

Few Data Available/Sparse Data

If only a minimal amount of data is available (i.e., the platform isinstalled freshly at the customer's premises), it is possible that thetechnique described above cannot find K nearest incidents, as they aresimply not yet present in the system. In such cases, a fallbackmechanism kicks in: for cases where an incident identical in the featurespace is found, its playbook is used for the recommendation of actions.If multiple identical incidents are found, their playbooks are fused bythe mechanism described above.

Large Clusters of Identical Incidents

In cases in which a tool is generating incidents automatically, it ispossible that hundreds or thousands of identical incidents (identical inthe position in the feature space, not necessarily in all data) aregenerated in the lifetime of the platform. In such cases, theseidentical incidents form a cluster in the space that overweighs allother incidents in the vicinity. If we pick K nearest incidents for anew incident that is near this cluster, all K incidents fall into thecluster, and thus there is no required variety of playbooks. In suchcases, we take from the cluster only a subset, which have uniqueplaybooks (i.e., playbooks with different sets of actions), Theremaining incidents from the cluster (with playbooks also identical tothe others in the cluster) are ignored and other, more distant neighborsare searched for. In other words, if there are more than (or equal to) Kincidents in the cluster, no additional incidents are searched for. Ifthere are fewer than K incidents in the cluster, additional incidentsare searched for.

FIG. 7 is a flowchart detailing the steps corresponding to identifying Knearest neighbors and then applying the different steps described aboveto the K nearest neighbors. In step 400, a playbook for an incident isrequested. If there are K incidents of the same type (step 402), the Kclosest incidents are found, by relevance (step 404), and any largeclusters are filtered (step 406) according to the technique describedabove; until enough relevant incidents are found or there are no moreincidents (step 408). The score of each action of each incident iscalculated using the techniques described above (step 410). If someactions have already been recommended (step 412) the already-recommendedactions are filtered out (step 414); otherwise the actions with theirscores are sent for comparison with a threshold (step 416).

If there were not K incidents of the same type (step 402), but there isan identical incident (step 418), the identical incident is used (step420) for computation of the score of each action in the incident (step410); otherwise an error message is returned (step 422).

User Feedback Integration

The user feedback to playbook recommendation is integrated in thefollowing way:

1) If the user does not pick the recommended playbook at all and picks adifferent playbook, the feature space with the incidents is enriched bythis selection. In other words, the selected playbook will be tied tothe incident and the recommended playbook will not be used. Insubsequent cases in which the technique is employed, the selection ofthe different playbook will be seen in the database and the actions ofthe different playbook will be considered according to the techniquedescribed above in connection with similar incidents, thus adapting tothe selection of the different playbook.

2) If the user picks a recommended playbook and removes some actions,the playbook is tied to the incident, which is, once closed, recorded inthe feature space. In subsequent recommendations, the removed actionshave a lower probability of being recommended. In other words, thefeedback concerning a user removing an action is inherently reflected inplaybooks generated in the future. In the technique described above, itcan be seen that the relevance of an action depends on the relativenumber of incidents in which the action was assigned to someone. Thus,since the action was removed from an incident's playbook, when in thefuture actions of this particular incident are used to synthesize a newplaybook the score of the action is affected by the fact that it wasremoved in the first place.

3) If the user picks a recommended playbook and adds some actions, theseactions are custom: They are thus treated with a special flag and forthe subsequent recommendations, these actions are shown irrespectivelyof their score (and by default selected to be played). The user willhave all manually added actions present in the recommended playbook.More precisely: When a custom action is manually added by a user, thesecurity incident response platform notifies the playbook generationsystem. Because custom actions are uniquely identified, they have oneundesired property: they can never occur in more than one playbook, andthus even though they are included in the final list of recommendedactions for an incident they could seem almost irrelevant because theyalways occur only once. To solve this problem, a list of manually addedactions are tracked and labelled as custom (treated with theabove-described special flag), thereby making it possible as a practicalmatter to present the custom actions in the final recommendation in theuser interface in a special fashion. However, the user can also removethe recommended actions from the playbook if he wishes so, and thedeleted custom actions do not need to be addressed in a special waybecause ordinary usage of the above-described machine learning techniqueaugments the data such that the removed or unused actions tend to stopappearing.

Performance

The performance of the technique described above has to be high enoughto allow real-time playbook assembly. The complexity of the technique islinear in the number of incidents and linear in the number of actions.The transformation of the incident into the feature space isstraightforward—the asymptotic complexity is of O(t*n) where t is thenumber of fields of the incidents and n is the number of unique valuesin case of cardinal types (such as enums).

The custom machine learning technique is computationally fast linear inthe number of incidents and the number of actions), and thus thereshould be no need to increase the performance. If needed, however,performance can be increased by reducing the number of potentiallyrelevant incidents and the number of relevant actions. If, for somereason, it becomes desirable to reduce the size of the sets, thesolution would be simple: 1) to optimize (minimize the size of) actionset A, it can be defined as a union of committed actions over allrelevant incidents; and 2) to optimize (minimize the size of) the set ofpotentially relevant incidents (not to iterate over all of them,however, only over a subset), the incidents can be arranged in aquadtree to have a spatial sorting mechanism in place and iterate bydistance.

Communication API

With reference to FIG. 8, the communication API allows the cybersecurityincident response system 700 to communicate with the playbook generationsystem 702 in a straightforward way. The user 704 has no means tocommunicate directly with playbook generation system 702. Rather, theuser interacts with the system through a web interface, which isconnected to the cybersecurity incident response platform. The platformsubsequently calls the playbook generation system, which runs as anindependent microservice on the very same virtual machine. The playbookgeneration system returns results to the cybersecurity incident responseplatform, which returns results to the user.

Some commands from user 704 trigger actions in cybersecurity incidentresponse system 700 that call playbook generation system 702, which inturn fetches data from database 706 and stores used playbook actions andplaybook customizations in database 706. The diagram of FIG. 8 capturesall relevant exchange of information. Playbook generation system 702 isa daemon that exposes a webservice that cybersecurity incident responsesystem 700 uses, in order to propose actions compatible with a newlycreated incident. The webservice exposes some REST methods tocybersecurity incident response system 700 but does not expose anythingto user 704.

There are two distinct activities by which the user can modify theattributes of an incident (thus the features of the incident): (1) bycreating a new incident, or (2) by updating one or more of theattributes (for example, some attributes might be missing at thebeginning and during the incident response, new facts are discovered andadded as attributes to the incident). The second activity arises insituations in which the user learns a new fact about the incident andupdates the incident. For example, imagine that the user finds out fromwhich IP the incident arose, and adds the IP into one of the CEF fields,which are transformed by the playbook generation system into features.In this case, the playbook generation system has to update featurevalues of the incident, after which the playbook generation system canpotentially present a new set of actions to be played.

The playbook generation system communicates proposed playbook actions tothe cybersecurity incident response platform using JSON (JavaScriptObject Notation), which is de facto a standard data format forinformation exchange via the REST application program interface.

During the playbook recommendation phase, the user is presented with alist of recommended actions together with the relevant incidents(determined using the K nearest neighbor technique describe above) fromwhich the actions were taken. The user can then remove or add some ofthe incidents (if, for example, the user does not consider them relevantor knows about an incident that might be important). The playbookgeneration system reflects the user's actions by recomputing theproposed playbook based on the updated set of incidents, according to aprocess described above.

The REST application program interface is defined as follows:

Info

GET/

-   -   Returns simple “hello world” message.

Init

POST/init/

-   -   Initializes playbook generation system, call s endpoint after        startup

Playbook

POST/generated_pb/

-   -   Generates playbook from given features

GET/incidents/{incident_id}/generated_pb/

-   -   Generates playbook using features from referred incident

POST/incidents/{incident_id}/generated_pb/

-   -   {generated_playbook_id}/    -   Acknowledges using the generated playbook

PUT/incidents/{incident_id}/closed/

-   -   Triggers recomputing model after closing an Incident

Actions

DELETE/playbooks/{playbooks_incidents_id}/actions/

{action_id}/

-   -   Notify about action removed from a playbook

POST/playbooks/{playbooks_incidents_id}/actions/{action_id}/

-   -   Notify about action added to a playbook

Blacklist

GET/blacklist/

-   -   Returns list of blacklisted actions

DELETE/blacklist/{action_id}/

-   -   Removes given action from blacklist

POST/blacklist/{action_id}/

Adds given action to blacklist

Projected data

GET/incidents/{incident_id}/actions/

-   -   Returns actions from playbooks on an incident used by playbook        generation system

GET/incidents/{incident_id}/features/

-   -   Returns features of an incident used by playbook generation        system

Settings

GET/settings/nearest_neighbour_count/

-   -   Returns number of nearest neighbours used to generate a new        playbook

POST/settings/playbook_neighbours/

-   -   Sets number of nearest neighbours used to generate a new        playbook

GET/settings/relevant_incident_age/

-   -   Returns the age in months after which a playbook is considered        to be less relevant

POST/settings/relevant_incident_age/

-   -   Sets the age in months after which a playbook is considered to        be less relevant

GET/settings/weights/

-   -   Returns weights of individual features used by playbook        generation system

POST/settings/weights/

-   -   Sets weights of individual features used by playbook generation        system

Stats

GET/stats/

-   -   Returns statistics of the playbook generation system machine        learning

Availability

GET/types/

-   -   Dictionary which shows the how many incidents of each individual        type the playbook generation system currently takes into account

GET/types/{incident_type}/available/

-   -   Returns True if we have enough incidents of given type, False        otherwise.

With reference to FIGS. 9-11, there are shown examples of displaysthrough which the user interfaces with the playbook generation systemthrough the cybersecurity incident response system. FIG. 9 shows ascreen display through which the user can notify the cybersecurityincident response system of the creation of an incident. FIGS. 10 and 11show screen displays through which details of a proposed playbook aredisplayed to a user. In the specific example of FIG. 19, the playbookgeneration system according to the invention has recommended a customplaybook for a denial-of-service incident that includes prescriptiveprocedures for restoring the affected system to its uninfected state. Inthe specific example of FIG. 11, the playbook generation systemaccording to the invention has recommended a custom playbook for aninsider abuse incident that includes prescriptive procedures for dealingwith internal information disclosed intentionally. In each instance, thecustom playbook is produced by the playbook generation system accordingto the techniques described herein, and the user may choose to use thecustom playbook, or use a different playbook, or remove an action to thecustom playbook, or add an action to the custom playbook, and theplaybook generation system records the user actions according to thetechniques described herein, so to affect future custom playbookrecommendations.

There has been described a cybersecurity incident response and securityoperation system and method employing playbook generation through custommachine learning. While several particular forms of the invention havebeen illustrated and described, it will be apparent that variousmodifications and combinations of the invention detailed in the text anddrawings can be made without departing form the spirit and scope of theinvention. Accordingly, it is not intended that the invention belimited, except as by the appended claims.

What is claimed is:
 1. A computer-implemented method comprising:encoding values associated with features of a plurality of cybersecurityincidents to obtain a feature vector for each cybersecurity incident,each cybersecurity incident having an associated playbook from aplurality of playbooks for responding to the cybersecurity incident;receiving details for a new cybersecurity incident, obtaining a newfeature vector for the new cybersecurity incident; selecting at leastone from the plurality of cybersecurity incidents based on the newfeature vector; determining a score for at east one action from theselected cybersecurity incidents; selecting one or more actions based onthe score of each action; creating a custom playbook for the newcybersecurity incident that includes the selected one or more actions;and causing presentation on a display of the custom playbook forresponding to the new cybersecurity incident.
 2. The method as recitedin claim 1, wherein selecting the at least one from the plurality ofcybersecurity incidents comprises: calculating a distance from the newfeature vector to each feature vector from the plurality ofcybersecurity incidents; and selecting one or more cybersecurityincidents that are within a predetermined distance from the new featurevector.
 3. The method as recited in claim 1, wherein each playbookcomprises at least one action for resolving the associated cybersecurityincident or for completing an investigation of the associatedcybersecurity incident.
 4. The method as recited in claim 1, whereindetermining the score comprises: calculating the score for the action asan average relevance of the action weighted by a commitment frequency ofthe action.
 5. The method as recited in claim 1, wherein the featurescorrespond to incident fields, wherein each incident field isconfigurable with a corresponding weight.
 6. The method as recited inclaim 1, further comprising: detecting a selection of a user of a newaction to respond to the new cybersecurity incident; adding the newaction to the custom playbook; and adding the new action to at least onefrom the plurality of cybersecurity incidents.
 7. The method as recitedin claim 1, wherein determining the custom playbook for the newcybersecurity incident comprises: including, in the custom playbook,actions with a score above a predetermined threshold.
 8. The method asrecited in claim 7, further comprising: providing a user interface WO toconfigure one or more thresholds for selecting actions to be added tothe custom playbook.
 9. The method as recited in claim 8, furthercomprising: detecting that multiple thresholds have been configured; andpresenting in the UI multiple playbooks, each of the multiple playbooksassociated with one from the multiple thresholds.
 10. A systemcomprising: a memory comprising instructions; and one or more computerprocessors, wherein the instructions, when executed by the one or morecomputer processors, cause the system to perform operations comprising:encoding values associated with features of a plurality of cybersecurityincidents to obtain a feature vector for each cybersecurity incident,each cybersecurity incident having an associated playbook from aplurality of playbooks for responding to the cybersecurity incident;receiving details for a new cybersecurity incident; obtaining a newfeature vector for the new cybersecurity incident; selecting at leastone from the plurality of cybersecurity incidents based on the newfeature vector; determining a score for at least one action from theselected cybersecurity incidents; selecting one or more actions based onthe score of each action; creating a custom playbook for the newcybersecurity incident that includes the selected one or more actions;and causing presentation on a display of the custom playbook forresponding to the new cybersecurity incident.
 11. The system as recitedin claim 10, wherein selecting the at least one from the plurality ofcybersecurity incidents comprises: calculating a distance from the newfeature vector to each feature vector from the plurality ofcybersecurity incidents; and selecting one or more cybersecurityincidents that are within a predetermined distance from the new featurevector.
 12. The system as recited in claim 10, wherein each playbookcomprises at least one action for resolving the associated cybersecurityincident or for completing an investigation of the associatedcybersecurity incident.
 13. The system as recited in claim 10, whereindetermining the score comprises: calculating the score for the action asan average relevance of the action weighted by a commitment frequency ofthe action.
 14. The system as recited in claim 10, wherein the featurescorrespond to incident fields, wherein each incident field isconfigurable with a corresponding weight.
 15. The system as recited inclaim 10, wherein the instructions further cause the one or morecomputer processors to perform operations comprising: detecting aselection of a user of a new action to respond to the new cybersecurityincident; adding the new action to the custom playbook; and adding thenew action to at least one from the plurality of cybersecurityincidents.
 16. A tangible machine-readable storage medium includinginstructions that, when executed by a machine, cause the machine toperform operations comprising: encoding values associated with featuresof a plurality of cybersecurity incidents to obtain a feature vector foreach cybersecurity incident, each cybersecurity incident having anassociated playbook from a plurality of playbooks for responding to thecybersecurity incident; receiving details for a new cybersecurityincident; obtaining a new feature vector for the new cybersecurityincident; selecting at least one from the plurality of cybersecurityincidents based on the new feature vector; determining a score for atleast one action from the selected cybersecurity incidents; selectingone or more actions based on the score of each action; creating a customplaybook for the new cybersecurity incident that includes the selectedone or more actions; and causing presentation on a display of the customplaybook for responding to the new cybersecurity incident.
 17. Thetangible machine-readable storage medium as recited in claim 16, whereinselecting the at least one from the plurality of cybersecurity incidentscomprises: calculating a distance from the new feature vector to eachfeature vector from the plurality of cybersecurity incidents; andselecting one or more cybersecurity incidents that are within apredetermined distance from the new feature vector.
 18. The tangiblemachine-readable storage medium as recited in claim 16, wherein eachplaybook comprises at least one action for resolving the associatedcybersecurity incident or for completing an investigation of theassociated cybersecurity incident.
 19. The tangible machine-readablestorage medium as recited in claim 16, wherein determining the scorecomprises: calculating the score for the action as an average relevanceof the action weighted by a commitment frequency of the action.
 20. Thetangible machine-readable storage medium as recited in claim 16, whereinthe features correspond to incident fields, wherein each incident fieldis configurable with a corresponding weight.